Multimedia creation, processing and delivery systems are rapidly appearing, fueled by technological advances and early standardization efforts. For example, improved processing and storage, transmission, encoding/decoding and other technologies are facilitating a shift from analog to digital broadcasting (i.e. television, cable, satellite and other distribution of digitized, and often digitally processed, multimedia data streams). Early standardization is further providing for interoperability and thus, less expensive, more readily available systems and components. Among the anticipated advantages of this shift are consumer access to a greater variety of higher quality programming and a high quality mass distribution medium for commercial industries.
Standards for encoding and decoding data streams comprising audio, video, other multimedia and non-multimedia information, include the internationally adopted MPEG-1 (ISO/IEC 11172-n) and MPEG-2 (ISO/IEC 13818-n) specifications, as espoused by the Motion Picture Experts Group. (MPEG-2 expands on MPEG-1 for broadcast-quality video and other applications.) Among other features, encoding/decoding specifications such as MPEG-1 and MPEG-2 (hereinafter collectively referred to as “the MPEG specifications”) prescribe an array of data stream format, timing, synchronization and device capability parameters essentially independently hardware/software system implementations. Stated alternatively, the MPEG specifications prescribe specific protocols and capabilities while allowing essentially unrestricted (and hopefully, continually advancing) systems for implementing the prescribed protocols and capabilities.
The MPEG specifications and other encoding/decoding standards have also been expanded to incorporate other emerging technologies. For example, the Audio Extension standard (ISO/IEC 13818-7) adds an AC-3 audio mode to MPEG-2. Further extensions within the MPEG-specifications have also been discussed, introduced and/or adopted as international standards for handling, facilitating and/or expanding the utility of data encoding, decoding, transmission and related issues. The MPEG specifications are hereby fully incorporated herein by reference as if repeated verbatim immediately hereinafter.
Unfortunately, while providing a growing common platform for achieving the above goals, the MPEG and related specifications are necessarily drafted, at least in part, through hindsight review of recent developments. Thus, despite revisions and expansions, they are also necessarily deficient with regard to certain unforeseen applications and other contingencies. MPEG-2, for example, failed to anticipate the growing requirements for complete digitally-encoded data stream splicing (“splicing”) support. Broadly stated, digitally-encoded data stream splicing involves appending a new data stream to a selected point in an existing data stream, thereby replacing the existing data stream at the splice point. With respect to broadcast media, splicing can be broadly viewed as switching from a currently broadcast data stream to a new data stream. For example, a need might exist for inserting a separately encoded commercial message into an ongoing movie broadcast (requiring a splice at the start of the commercial and a further splice back to the movie at the end of the commercial), among other applications.
The difficulties associated with splicing together digitally encoded data streams are more easily understood by comparison with the more straight forward tasks of splicing together analog or even non-encoded digital data streams. For example, non-encoded data streams are typically transmitted, received, buffered by a receiver and displayed in the same order. Further, each individually-presented data portion or “frame” of a non-encoded data stream typically contains a similar, if not identical, amount of data. Splicing together two analog data streams or two non-encoded digital data streams therefore essentially involves switching from one stream source to a second data stream source at a readily determinable frame boundary. Systems and methods for performing such splicing are also well known in the art.
In contrast, frames of a digitally encoded data stream are typically received by a decoder in a different order than they are displayed. In fact, different encoders typically produce data streams having vastly different frame and informational characteristics. Further, a primary goal of encoding is to compress the data stream so that the data stream will require less transmission bandwidth and less storage space, among other advantages.
Thus, an MPEG-2 encoded video data stream will contain only a limited number of frames containing complete data for independently re-creating a corresponding image (commonly referred to as “I” frames); other frames, commonly referred to as predicted frames (“P-frames”) and bidirectional frames (“B-frames”), contain a lesser, variable amount of reference data for re-creating a video image by comparison with previously decoded frames. In addition, each data stream must be encoded such that (during decoding) a decoder buffer will continuously contain a sufficient number of frames for continuous decoding, but not so many frames as to overflow the decoder buffer.
Splicing together digitally encoded data streams is further complicated by the independent nature of each encoded data stream. In addition to the independent form and synchronization data associated with each data stream, conventional encoder and decoder specifications fail to provide requisite timing or splice point parameters. Such difficulties are further exacerbated by the lack of data available for relating one stream to another. For example, no parameters are provided for directly and reliably determining the contents of a decoder buffer occupied by data from more than one stream. Thus, there is no direct support provided for assuring that a decoder buffer will contain a sufficient amount of data from each of two streams to be spliced together such that continuous data flow will be sustained and the buffer will not underflow or overflow.
Despite the above and other difficulties relating to splicing together encoded data streams, solutions have been proposed, two of which are already in use. One conventional solution is to decode the subject data streams, perform the splice in decoded form and then re-encode the data streams as a single data stream, thereby effectively avoiding the inherent problems of splicing encoded data streams altogether. Unfortunately, encoding and decoding hardware that is sufficiently robust to perform the splice in real time is also expensive. In broadcasting applications, each “television channel” is provided by a separate data stream source, any one of which might well require splicing. Therefore, replacement of multiple encoders and decoders will likely be required.
According to another conventionally utilized method for splicing encoded data streams, blackened and muted video frames (“leader frames”) are appended to the start of the new data stream. Among the advantages of this method is that a minimal amount of data is required to produce the leader frames. Therefore, decoder buffer overflow will not occur. In addition, the leader frames act to mask visual artifacts resulting from a less than precise splicing of a data stream pair, such as a loss of display synchronization commonly referred to in the art as rollover. Further, such a method can be performed using existing encoders and decoders, thereby avoiding replacement costs. Unfortunately, the splice is not transparent. Blackened frames are noticeable to a consumer presented with the displayed data stream. Since the leader frames form a part of a new stream, such as a commercial message, the lost “air time” is of particular concern to the advertiser who may pay in excess of $900,000.00 for a 30 second commercial. (Each frame might therefore cost in excess of $1,000.00.)
Despite these attempts, standardization efforts currently under proposal by the Society of Motion Pictures and Television (“Proposed SMPTE Standard for Television—Splice Points for MPEG-2 Transport Streams”), suggest abandonment of current splicing methodologies in favor of a new extension to the MPEG-2 specifications. The proposal suggests a new, integrated protocol directed specifically at performing transparent or “seamless” splicing of a data stream pair. Broadly stated, the proposed standard provides for incorporation of splice points and related splicing-support information directly into the data stream at the time of encoding. Unfortunately, the proposed extensions will at least require the expense of replacing existing encoders. To make matters more difficult, the proposed standard is not yet ratified. Therefore, potential modifications, as well as inevitable varied interpretations, implementations, incompatibilities, proprietary “further extensions” and other issues are likely to arise.
Accordingly, there is a need for an apparatus and methods for splicing together digitally encoded multimedia data streams without requiring new encoders or decoders, and without introducing noticeable and air-time consuming artifacts. There is further a need for such apparatus and methods that avoid a requirement for modification of conventional encoding/decoding specifications.